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I. INTRODUCTION 


In today's dynamic and interconnected world, the safety 
and efficiency of transportation systems stand as paramount 
concerns. Road sign recognition, a fundamental component 
of intelligent transportation systems, plays a pivotal role in 
enhancing road safety, facilitating efficient traffic 
management, and enabling the progression towards 
autonomous driving. Recognizing and interpreting road 
signs is crucial for providing timely and accurate 
information to drivers and autonomous vehicles, aiding in 
adherence to traffic regulations, and ensuring safe 
navigation on roadways. 


Road signs convey essential messages to drivers, 
alerting them to speed limits, warnings about hazards, 
providing directions, and communicating regulatory 
instructions. The ability to recognize these signs accurately 
and swiftly is imperative, as it directly influences driver 
decision-making, reduces accident risks, and contributes 
significantly to the overall efficiency of transportation 
networks. 


Traditionally, road sign recognition relied on manual 
interpretation by human drivers. However, advancements in 
computer vision, machine learning, and neural network- 
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Abstract— This paper presents an in-depth study of road sign recognition 
techniques leveraging neural networks. Road sign recognition stands as 
a critical component of intelligent transportation systems, contributing to 
enhanced road safety and efficient traffic management. The paper focuses 
on exploring various neural network architectures for example, 
backpropagation neural network and hybrid neural network which is a 
combination of two neural network (backpropagation neural network and 
bidirectional associative memory), training methodologies, dataset 
considerations, and performance evaluations for accurate and real-time 
recognition of road signs. The experimental result shows that the hybrid 
neural network is faster than the backpropagation neural network in the 


completion of the training process with higher recognition accuracy. 


based approaches have revolutionized this field. These 


technologies enable automated recognition and 
interpretation of road signs from images or video streams 


captured by cameras mounted on vehicles or infrastructure. 


The complexity of road sign recognition arises from 
various factors, including diverse environmental conditions, 
variations in sign appearances due to aging, damage, or 
regional differences in designs and symbols, as well as the 
need for real-time processing to ensure timely responses. 
Overcoming these challenges requires sophisticated 
algorithms, robust training methodologies, and extensive 
datasets that encompass the diversity of road signs 
encountered in different geographical locations and 
environmental conditions. 


In recent years, there has been notable research 
advancement in the domain of road sign recognition. 
Namyang and Phimoltares [1] utilized a combination of 
Support Vector Machines (SVM) and Random Forest 
algorithms, along with HOG and the Color Layout 
Descriptor (CLD), for traffic sign classification. Soni et al. 
[2] employed HOG and LBP descriptors with Principal 
Component Analysis (PCA) and Support Vector Machines 
(SVM) for traffic sign classification. Sapijaszko et al. [3] 
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proposed a traffic sign recognition system encompassing 
stages such as normalization, feature extraction, 
compression, and classification. Aziz and Youssef [4] 
proposed a traffic sign recognition system utilizing feature 
extraction and the Extreme Learning Machine (ELM) 
algorithm. Wang [5] introduced a traffic sign classification 
system employing three machine learning classifiers: 
Logistic Regression (LR), Multilayer Perceptron (MLP), 
and SVM. 


But, neural networks, particularly Backpropagation 
Neural Network, have demonstrated remarkable capabilities 
in image recognition tasks, making them a promising 
approach for road sign recognition. This paper outlines the 
performance evaluation of the backpropagation neural 
network and the hybrid neural network. 


Il. PRELIMINARIES 
A. Road Signs 


Road signs serve as crucial elements of traffic control 
and safety, providing essential information to drivers, 
pedestrians, and other road users. These signs are 
standardized visual cues that communicate various 
instructions, warnings, regulations, and guidance about road 
conditions. Figure 1 shows some road signs with their 
corresponding meanings. 


Signs Al Meaning 
“id School zone 
Crossroad 
abead 
ry k 
Q ans 


Fig. 1 Road signs 


B. Neural Networks 


A neural network is characterized as a reasoning model 
inspired by the human brain's structure. The brain is 
composed of a highly interconnected network of nerve cells 
known as neurons, which serve as the fundamental units for 
processing information. Human brains encompass nearly 
10 billion neurons and an extensive network of 60 trillion 
synapses interconnecting them, as documented by Shepherd 
and Koch [6] Leveraging the simultaneous activation of 
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multiple neurons enables the brain to execute tasks at a 
considerably higher speed compared to today's fastest 
computers. 


Despite their straightforward architecture, a collection 
of neurons yields remarkable computational capabilities. 
Each neuron comprises a cell body, a multitude of dendrites 
(fiber-like structures), and a solitary elongated fiber 
identified as the axon. Dendrites form an intricate network 
surrounding the cell body, while the axon extends towards 
other neurons' dendrites and cell bodies. Refer to Figure 2 
for an illustrative representation of a biological neuron. 


Synapse Axon 
X. 


+ a 
Dendrites Soma 


Fig. 2 Biological neuron 


Our brain functions as an intricate and sophisticated 
information-processing system that operates in a highly 
complex, nonlinear, and parallel manner. Unlike traditional 
systems where data processing occurs in specific areas, in 
neural networks, information is stored and processed 
simultaneously across the entire network. This global 
approach to both data and its processing distinguishes 
neural networks by their widespread rather than localized 
functionality. The adaptability of connections between 
neurons, causing variations that contribute to arriving at the 
‘correct’ outcome, highlights the plasticity of neural 
networks. Consequently, these networks possess the 
capacity to learn from experiences, marking learning as a 
foundational and vital attribute of biological neural 
networks. The innate ability to learn effortlessly prompted 
endeavors to replicate a biological neural network's 
functionality within a computer environment. 


C. Backpropagation Neural Network 


The backpropagation neural network (BPNN) 
architecture comprises an input layer, an output layer, and 
one or more hidden layers. The number of input units 
corresponds to the quantity of bits or values representing the 
input pattern, accommodating these n bits or values. 
Similarly, the count of output units is determined by the bits 
or values associated with the output pattern. 
Conventionally, the network establishes complete 
connectivity exclusively between adjacent layers, forming 
fully connected pathways throughout the network as shown 
in Figure 3. 
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Hidden 


Input 


Output 


Fig. 3 Backpropagation Neural Network 


The backpropagation method uses two steps: 


1. In the ‘forward step’, the input is applied and 
allowed to propagate to the output. The error 
values of the output units are calculated by 
subtracting output value from target value for each 
unit. 

2. In the ‘backward step’, errors are propagated 
backwards, and weights are modified. 


The network's training objective involves refining the 
weights to ensure that a given set of inputs yields the 
intended set of outputs. For conciseness, these input-output 
combinations are often denoted as vectors. Training 
operates on the premise that each input vector aligns with a 
target vector, symbolizing the anticipated output; 
collectively, these form a training pair. Typically, a network 
undergoes training with multiple such pairs. For instance, 
an input pair might encompass a sequence of ones and zeros 
representing a binary image corresponding to an alphabet 
letter. The compilation of these training pairs constitutes a 
training set. 


D. Bidirectional Associative Memory 


Bart Kosko [7] introduced Bidirectional Associative 
Memory (BAM) as a heteroassociative neural network as 
shown in Figure 4. It operates by receiving an input pattern 
represented as a vector across one group of neurons and 
generates a correlated yet distinct output vector across 
another set, and conversely does the same in reverse. 
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Input Layer 


Output Layer 


Fig. 4 BAM Network Architecture 


The main features of a BAM are given below: 


e A BAM comprises two layers of interconnected 
neurons. 

e Neurons within one layer establish complete 
connections with neurons in the other layer. 

e There are no interconnections among neurons 
within the same layer. 

e The storage capacity and reliability of recall hinge 
on the network architecture and the algorithms 
used for both recalling and learning. 

e Enhancing performance can be achieved by 
introducing additional layers or establishing more 
interconnections among neurons. 


The input to a BAM network is a vector of real number, 
usually in the set {—1, +1}. The output is also a vector in 
the set {—1, +1} with the same or different dimension. 
These vectors can be considered as patterns, and the 
network makes heteroassociation of patterns. If the output 
is required to be the same as input, then the network is said 
to make auto-association. 


BAM Lape: BPNN Layer 


Fig. 5 Hybrid neural network 


E. Hybrid Neural Network 


A hybrid neural network consists of two distinct neural 
networks: the BAM neural network and the Bidirectional 
neural network. BAM is employed to reduce the dimensions 
of the feature matrix, thereby enhancing the speed and 
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efficiency of recognition. Figure 5 illustrates the network 
architecture for this hybrid neural network. 


IW. EXPERIMENTS AND PERFORMANCE 
EVALUATION 


Recognition of Road Sign is a step-by-step processing 
of road sign. These processes include: 


e Preprocessing 
e Recognition of sign 
e Performance evaluation 


The effectiveness of the algorithm has been justified for 
different Road Sign images of different resolutions. The 
algorithm is capable of preprocessing and recognizing signs 
of any grayscale images. The implementation of the 
algorithm was carried out using the C programming 
language. 


At the beginning, a sign is chosen. The sign is a 
grayscale image. Whole of the task is done by the following 
way: 

A. Preprocessing 


Road signs are being preprocessed by following a 
sequence of operations: (Capturing, Binary image 
conversion. 


Capturing: Road sign acquisition involves utilizing a 
camera for capturing purposes. For this study, specific 
obligatory road signs were utilized to create an image 
database stored as BMP type files. As part of the capturing 
process, standardization, and geometric normalization, 
involving adjustments in size and direction, were applied to 
the images. For analysis purposes, the images were resized 
to a resolution of 64 x 64 pixels, as depicted in Figure 6. 


Fig. 6 Road sign image 


Binary Image Conversion: The initial image was notably 
in color. It underwent a transformation into a grayscale 
image and subsequently underwent binarization, a process 
chosen for its simplicity in pattern matching during sign 
recognition. Each sign is represented as a matrix of 
numerical values, ranging from 0 to 255, which can be 
further translated into binary format (Os and 1s). The 
conversion sequence is demonstrated in Figure 7. 
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Grayscale 


Image 
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seun 
pozuemg 


Fig. 7 Image conversion 


The road signs are arranged within a 64 x 64 grid and 
inputted into the neural network's input layer as feature 
vectors or training patterns. Consequently, the grid 
configurations are depicted as vectors comprising 4096 
components (where each vector component is 1 if the pixel 
in the grid is shaded, otherwise it is 0). The hidden layers 
encompass 60 neurons each, which account for 60% of the 
input layer. Considering a total of 64 signs, inclusive of 
both mandatory and other signs, the output layer comprises 
64 neurons. Having 64 neurons in the output layer enables 
the representation of a 6-bit code (2^6 = 64) for classifying 
each target output. Hence, the target outputs range from 
000000 to 111111, aligning with each distinct sign. 


B. Recognition of sign 


The road sign recognition phase is most important and 
complicated phase. Hence, the sign is 64 X 64 pixels 
image. Each sign has a feature matrix of 4096 elements in 
it. Each element is nothing but binary values (0 and 1). 


eect 


111 AARAAA 


Fig. 8 Recognition of a sign "NO PARKING” 


Hence, in this experiment the number of neurons in 
input layer is 4096, neurons in input layer for BPNN and 
output layer for BAM is 16, number of neurons in hidden 
layer is 10, and finally the neurons in output layer is 6. The 
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number of neurons in hidden layer can vary from 50% to 
70% of its input neurons. 


Figure 8 shows the snapshot of the program output of a 
normal image. 


C. Performance evaluation 


To assess the neural network's performance, a series of 
experiments were conducted, employing separate training 
and test image sets for each sign without any overlap 
between them. The back-propagation neural network 
underwent training utilizing default learning parameters 
(learning rate 0.3, threshold 1) over 75 epochs. 
Subsequently, the network was employed to recognize 
individual signs. 


Throughout the training process, the program continued 
execution until the error reached a minimum threshold 
level, illustrating the error reduction per iteration in a 
graphical representation. Initially, the task was 
implemented using the BPNN algorithm alone, followed by 
merging the BAM and BPNN algorithms to train and 
recognize road signs. Upon analysis, it was observed that 
employing the hybrid network (BAM and BPNN) required 
fewer iterations for training and less time for sign 
recognition compared to BPNN alone. 


Table I Iteration Vs. Error (70%) 


Iteration | BP (70%) BP With BAM (70%) 
2 3,866509 1,959864 
5 3,858203 1,894699 
10 3,503066 1,881338 
100 3,435648 1,702669 

500 3,217658 1,119279 
1000 3,095643 0,612061 
1500 2,943532 0,575292 
2000 2,873423 0,554884 
2500 2,806753 0,543144 
3000 2,645987 0,53713 
3500 2,546534 0,531881 
4000 2,513423 0,492877 
4500 2,485645 0,315097 
5000 2,437864 0,293473 
6000 2,376588 0,278986 
7000 2,238757 0,272395 
8000 2,187658 0,268759 
9000 2,074542 0,266053 
10000 1,984532 0,264022 
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15000 1,912328 0,258816 
20000 1,813421 0,256447 
25000 1,746574 0,255095 
30000 1,698342 0,25338 

34075 1,607801 0,009981 


40000 1,576457 
45000 1,53768 

50000 1,457854 
55000 1,356245 
60000 1,247856 
65000 1,109854 
70000 1,065242 
75000 0,176542 
80000 0,039947 
85000 0,017469 
88043 0,009997 


Iteration Vs. Error 


5 

4 
5 3 —o— BP 
wu 2 

1 . 

0 =E BP With 

BAM 
0 50000 100000 
Iteration 


Fig. 9 Iterations Vs. Error (70%) 


The training process involved eight training input 
patterns and employed an error threshold (e.g., 0.001) to 
halt training. The number of iterations was contingent upon 
the percentage of the hidden layer and the algorithm utilized 
during training. For instance, setting the hidden layer to 
70% of the input layer resulted in 88043 iterations, while 
reducing the hidden layer to 50% led to 43067 iterations. 


Upon adopting the hybrid network with the same 
percentage of the hidden layer, the iteration count decreased 
further. Specifically, it decreased from 34075 to 14977, 
illustrating an even more pronounced reduction in iterations 
compared to previous results. 
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Table 2 Iteration Vs. Error (50%) 
Iteration | BP (50%) BP With BAM (50%) 
2 6,933186 3,07014 
5 5,694922 3,055324 
10 4,678283 3,025566 
100 4,333929 1,374816 
500 4,045357 0,567401 
1000 3,874532 0,316263 
1500 3,523548 0,134039 
2000 3,223324 0,077653 
2500 2,963092 0,057539 
3000 2,927669 0,04723 
3500 2,906451 0,040717 
4000 2,566751 0,035315 
4500 2,550943 0,031443 
5000 2,539485 0,028615 
6000 2,530225 0,023834 
7000 2,522263 0,020596 
8000 2,515119 0,018211 
9000 2,502996 0,016202 
10000 2,448071 0,014673 
14977 2,056355 0,01 
20000 1,800224 
25000 1,034644 
30000 0,176542 
35000 0,039947 
40000 0,017469 
43067 0,009997 
Iteration Vs. Error 
10 
S 5 —eBP 
wi 
E —E-—BP With 
O 20000 40000 60000 BAM 


Iteration 


Fig. 10 Iterations Vs. Error (50%) 
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Comparing the performance of two networks as BPNN 
and hybrid neural network a decision had been taken that 
the hybrid neural network takes less iteration than BPNN in 
completion of the training process. 


IV. CONCLUSION 


Neural network-based road sign recognition holds 
immense potential in revolutionizing transportation systems 
by enhancing road safety and traffic management. The 
paper underscores the significance of neural networks in 
this domain and outlines the performances of BPNN and 
hybrid neural networks with experimental results. And the 
result shows that the hybrid neural network performs faster 
than BPNN with high recognition accuracy. 
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